Influence of Inter-Annotator Variability on Automatic Mitotic Figure Assessment

Density of mitotic figures in histologic sections is a prognostically relevant characteristic for many tumours. Due to high inter-pathologist variability, deep learning-based algorithms are promising solution improve tumour prognostication. Pathologists the gold standard database development, however, labelling errors may hamper development accurate algorithms. In present work we evaluated benefit multi-expert consensus (n = 3, 5, 7, 9, 11) on algorithmic performance. While training with individual databases resulted highly variable F$_1$ scores, performance was notably increased and more consistent when using three annotators. Adding annotators only minor improvements. We conclude that by few pathologists label accuracy be best compromise between time investment.